What does this PR do?

Adds a new util to parse a new modeling_* file, whether or not it is from transformers, and lists the closest possible matches from existing models in the library using embeddings from Qwen3.

Why?

Because most models should be able to be defined as modulars from each other. It's not exactly the case yet, see this space to see the existing links between models and the ones yet to be included.

Also, there are more than 400 models. We cannot remember where is what. Hopefully this will accelerate development/porting of new models into a modular format and the modernization of older, non-modular models.

How do I use it?

If you're adding a new model to transformers and have already some form of runnable modeling file, the only action is (with a sufficiently powerful machine) to run (here from #39534 that adds beit3 model) from the transformers repository:

python utils/modular_model_detector.py --modeling-file /path/to/your/modeling_file.py 

will get you the following (excerpt):

stuff.py::Beit3ImageTextMatchingOutput:                                      
  embedding:
    blip_2::Blip2ImageTextMatchingModelOutput (0.9994)
    chinese_clip::ChineseCLIPOutput (0.9818)
    metaclip_2::MetaClip2Output (0.9817)
    owlv2::Owlv2Output (0.9817)
    aimv2::Aimv2Output (0.9817)
  jaccard:
    owlv2::Owlv2Output (0.9667)
    metaclip_2::MetaClip2Output (0.9667)
    altclip::AltCLIPOutput (0.9667)
    owlvit::OwlViTOutput (0.9667)
    blip::BlipOutput (0.9667)
  intersection:
    metaclip_2::MetaClip2Output
    owlv2::Owlv2Output

stuff.py::Beit3MLP:
  embedding:
    efficientloftr::EfficientLoFTRMLP (0.9717)
    seggpt::SegGptMlp (0.9650)
    mgp_str::MgpstrMlp (0.9645)
    vitpose_backbone::VitPoseBackboneMLP (0.9640)
    granitemoehybrid::GraniteMoeHybridMLP (0.9634)
  jaccard:
    chinese_clip::ChineseCLIPTextSelfOutput (0.5294)
    convbert::ConvBertSelfOutput (0.5294)
    bert::BertSelfOutput (0.5294)
    roformer::RoFormerSelfOutput (0.5294)
    layoutlmv3::LayoutLMv3SelfOutput (0.5294)
  intersection:

stuff.py::Beit3FeedForwardNetwork:
  embedding:
    prophetnet::ProphetNetFeedForward (0.9764)
    dab_detr::DabDetrDecoderLayerFFN (0.9729)
    kosmos2::Kosmos2TextFFN (0.9698)
    kosmos2_5::Kosmos2_5TextFFN (0.9698)
    parakeet::ParakeetEncoderFeedForward (0.9677)
  jaccard:
    groupvit::GroupViTMLP (0.4898)
    convbert::ConvBertOutput (0.4600)
    chinese_clip::ChineseCLIPTextOutput (0.4565)
    bert::BertOutput (0.4565)
    roformer::RoFormerOutput (0.4565)
  intersection:
....

From experience: Jaccard-based indices are lower in value because they punish more harshly token differences but are still useful. Embeddings scores calibration is a bit effy - a score below 0.90 is definitely on the lower end. The intersection exists to guide the user a bit more.

In any case, the command above will take care of downloading the precomputed indices from the corresponding hub dataset - we will need to setup a scheduled job to do it from time to time (like weekly?).

Should you want to recompute the index, simply run

python utils/modular_model_detector.py --build

Where you can change the model ID to try other models for instance. HF members can also push to the dataset, should you find better embeddings/matches.

This is a relatively minimal API, I didn't want to over-engineer it. Compared to the space above where I link models together, the main difference is that here I'm linking functions, methods and Classes together rather than whole models.